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ABSTRACT 



The Operation with Economic Efficiency (OEE) Program in the Republic of 
Korea Air Force (ROKAF) was designed to reduce the cost and improve the efficiency 
of all ROKAF operations. The ROKAF is now faced with the problem of developing 
a more efficient pilot candidate selection and training program which will improve pilot 
quality and reduce attrition rates and the number of aircraft accidents. The present 
research provides a starting point for the development of a more efficient pilot 
selection and training program. If successful this program should result in the selection 
of pilot candidates exhibiting a significantly higher probability of completing the pilot 
training program. This thesis also provides a summary of aviation selection test 
batteries presently in use by the U.S. Air Force and U.S. Navy and applies the 
principles of decision theory in determining whether a change in the ROKAF's present 
pilot selection system is feasible. 
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I. INTRODUCTION 



A. BACKGROUND 

Throughout the history of aviation there has been constant concern with 
selecting those individuals with both mental and physical qualities desirable for 
optimum performance in flight environments. The intensity and quantity of research 
performed over the years in the particular area of pilot selection highlights the fact that 
the key to combat air superiority is the selection of high quality pilot candidates. 
During World War II and the Korean conflict it became further apparent that the 
ability to gain total tactical and strategic advantage over the enemy depended on 
whether or not air superiority was achieved. Fortunately the U.S. was able to achieve 
this superiority in the air during the last three major conflicts. Unfortunately there is a 
growing consensus among military analysts that the U.S. may be lulling itself into a 
sense of complacency. The following quotation from a publication by B. K. Holloway 
underscores the basis for this growing complacency: [Ref. lj 

• "Twenty-six million living Americans are veterans of military service, and most 
of them have served in wartime. How many of these 26 million ever had to face 
an enemy who held air superiority ? ... In all, probably no more than one out 
of 150, for after February 1943 the U.S. and allies had undisputed air 
superiority in the Mediterranean and Western Europe; after August of 1943 we 
had it in the Pacific... In Korea we won air superiority twice - from the North 
Korean Air Force in the first two months of that war, and again from the 
Chinese Air Force after November 1950. In South Vietnam, our air superiority 
has come by default. In North Vietnam it has yet to be seriously challenged." 

Today we have reached an age of strategic parity where Eastern block 
technological developments in air combat systems are advancing at a rate comparable 
with those as the West. The key in future conflicts thus becomes improved combat 
effectiveness and thus, ultimately, the development of improved aircrew selection and 
training programs. 

The essence of these programs might be summarized simply as: 

1) Using an effective selection method, find the best candidate pilot, 

2) Provide him/her with the best and most cost effective training, 

3) Significantly improve retention rates to maximize the training investment. 
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The first step, selecting the best pilot candidate, is not an easy one. The key 
questions become, "What mental and physical characteristics do we look for ?" and 
"What are our selection criteria ?" The answers to these two basic questions have been 
the object of much of the pilot candidate selection research, which will be addressed in 
the following section. 

The ROKAF is currently experiencing an excessive and unacceptable attrition 
rate in its pilot training programs. This problem is further compounded by the fact 
that the total number of candidates admitted to the flight training program has steadily 
increased since the early 1980's, while training resources remained limited. As a result 
the candidates who remain in the program are afforded fewer hours of instruction in 
the air and thus do not receive the desired quality or quantity of training. 

In the early 1960's both the U.S. Navy and Air Force flight training programs 
suffered similar attrition problems. Figure 1.1 provides a summary of the attrition 
categories for the respective services at that time and Appendix A provides a detailed 
description of attrition categories. The comparison of attrition rates between USN and 
USAF presented in Figure 1.1 as a result of various factors. USN had 60.5% attrition 
rate and 21.5% of that was due to flight failure. Similarly USAF had 49.0% due to 
flight failure out of 33.8% attrition. However, with the significant improvement in 
existing selection methods and the incorporation of new methods of psychological 
screening both services were able to decrease their respective attrition rates by almost 
30% by 1977 [Ref. 2]. 

The present research attempts to summarize some of these improved selection 
methods utilized by both the U.S. Navy and Air Force and presents recommendations 
based on this summary for improving the pilot selection program of the ROKAF. 

B. OBJECTIVE 

The Korean Air Force (ROKAF) needs a more effective way to select and predict 
performance of pilot candidates for success in primary flight training and operational 
flight training to maintain its air superiority. The purpose of this thesis, therefore, is to 
suggest a pilot selection system which selects potential successes in flight training and 
improves pilot quality. This research also provides a) an overview of pilot selection 
research, b) testing methods, and c) a related decision theory and cost effectiveness 
model. 
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USN 
60.5 % 



USAF 



33.8 % 



Motivational Attrition 



NOM 18.4% 


DOR 1 42.1% 


SIE 22.4% 


MOA 11.4% 


(Not Officer 
Material) 


(Drop on Request 
or Voluntary 
Withdrawal) 


(Self 

Initiated 

Elimination) 


(Manifestation 
of Apprehension) 



Flight 



FF 21.5% 
(Flight Failure) 




FD 49.0% 

(Flying Deficiency) 




Medical 


NPQ 14.9% 




M 12.3% 


(Not Physically 
Qualified) 




(Medical) 




Academic 


2.0% 




2.9% 




Other 


1.1% 




2.0% 



* includes 3.6% Air Sickness. 



Figure 1.1 Representative Navy and Air Force Pilot Attrition Categories (From Ref.2). 
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II. U.S. PILOT SELECTION RESEARCH 



A. WORLD WARS 

It was first noted during WW I that a significant number of aircraft accidents 
were due not primarily to aircraft failures but to human error. Thus, based on this 
data France, Italy and England all initiated programs to aid in the prediction of pilot 
success during primary flight training. U.S. efforts began as early as 1919. [Ref. 3] 
Early selection tests in U.S. were based primarily on Paper-and-Pencil 

Perceptual/Cognitive tests with psychomotor devices and were able to predict flight 
performance of pilot candidates with resonable validity. 

The development of selection tests continued between the First and Second 
World Wars. Much of this effort was aimed at the development of psychomotor test 
devices which simulated stick and rudder movements of the aircraft. 

Perceptual/Cognitive Paper-and-Pencil tests likewise continued to be developed but 
both these and the psychomotor tests lacked significant predictive validity. A 
contributor to this fact was the deficiency in external criteria of in-flight performance. 
During W'W II the demand to train large numbers of pilots in a short amount of time 
along with increased training cost created the necessity for minimizing attrition due to 
poor proficiency. The Navy thus incorporated in its selection test battery numerous 
items previously evaluated in the civilian pilot training program. In the "Pensacola 
1000 aviator study", evaluation was made of the validity of more than 60 psychological, 
physiological and psychomotor test items from the Navy selection test battery utilizing 
over 900 Navy flight candidates. Results indicated that psychological and psychomotor 
measures exhibited greater validity for the prediction of success in flight training than 
did physiological measures and verified the validity and effectiveness of these 
psychological instruments and psychomotor tests in the prediction of flight success. 
(Ref. 3] 

The US Army and Air Force also developed their pilot selection testing battery 
during WW II and several useful methods such as the Complex Coordination test and 
Divided Attention test are still used in the present selection test battery. The Complex 
Coordination test, similar to USN one, used by the requires the candidate to make 
simple controlled movements of a stick and rudder in response to patterns of visual 
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stimuli. During WW II it was the most useful and. most highly weighted test. Other 
testing methods developed included the Finger Dexterity, Rudder Control Test and 
Two Hand Pursuit Test. Typical apparatus test validity coefficients for pilots, 
navigators and bombardiers are presented in Table 1 [Ref. 4]. Note that the test most 
highly related to pilots is the Complex Coordination Test. The Discrimination 
Reaction Time is significantly related to navigators and bombardiers. 



TABLE 1 

VALIDITY OF APPARATUS TESTS 



(year, sample size) 



Complex Coordination 


Pilots 


Navigators 


Bombardiers 




(1943, 3151) 
r = .33, r c =.40 


(1942, 1022) 
r = .17, r c =.24 


(1943, 1829) 
r = .10, r c =.13 


Rotarv Pursuit with 
Divided Attention 


(1943, 3146) 
r=.l4, r c =.22 






Finger Dexterity 


(1943, 4779) 
r = .07, r c= 10 


(1943, 1021) 
r = .10, r c= 13 


(1943, 1828) 
r= .13, r c _ 15 


Discrimination 
Reaction Time 


(1943, 4779) 
r = .25, r c =.28 


(1942, 1022) 
r = .27’ r c =.35 


(1942, 1829) 
r= .22, r c =.25 


Rudder Control Test 


(1943, 3146) 
r = .22, r c =.30 






Two Hand Pursuit 


(1943, 1385) 
r = .27, (average) 






Two Hand 
Coordination 


(1943, 4779) 
r = .31, r c =.35 


(1942, 1022) 
r = .26, r c = .29 


(1943, 1828) 
r = .09, r c = .12 



r = validity coefficient based on dichotomous pass/fail training criterion. 
r c = validity coefficient corrected for restriction in range. 

(From Reference 4) 
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B. POSTWAR RESEARCH 



Psychomotor Tests were used in USAF selection battery until 1951 but were 
deleted due to administrative, quality control and reliability problems. A subsequent 
study of the revision of the psychomotor tests indicated that the use of simple motor 
skill tests offered little if any predictive value for flight school success. It was 
concluded that this was due to: [Ref. 5] 

1) The unreliability of both predictor and criterion scores, 

2) The use of inappropriate motor tests for the particular criteria being 
evaluated, and 

3) The task of flying an airplane is too complex for simple motor skill tests to be 
of substantial benefit. 

This indicated that other more complex abilities were necessary for successful 
pilot performance and that the goal of future research should be to: 

1) Define the abilities, 

2) Quantify the basic behaviors making up the abilities, 

3) Devise accurate test measures. 

These problems were considered continuously through the 1970s. As a result, 
currently the USAF has been achieving great success in predicting flight performance 
in its UPT program. This success will be addressed in more detail in Chapter III. 

The other type test which has been the most widely and easily used as a selection 
test has been the Perceptual/Cognitive Paper- and- Pencil tests. The U.S. military 
service paper-and-pencil test battery consisted of : [Ref. 3] 

1) A general intelligence component composed of verbal and quantitative items, 

2) Mechanical comprehension, 

3) A spatial component, and 

4) A background or biographical inventory composed of miscellaneous subset of 
items usually of an historical nature known to relate to aviator success. 

The analysis of EEG (Electroencephalogram) recordings has also received much 
attention as a possible predictor of aircraft accidents. It has been found that 
individuals with abnormal EEG's are involved in accidents at three times the rate of 
those individuals with normal EEG's 1 [Ref. 3]. As a result of these and similar findings 
the Navy now screens all new student aviators for abnormal EEG's prior to 
commencement of flight training. 



1 These findings have resulted in the Danish Air Force's use of EEG recordings in 
the selection of pilot candidates. 
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C. CURRENT RESEARCH 



The establishment of reliable criterion measures is a prime factor in any selection 
process. This continues to be an extremely difficult and dynamic task. For example, 
the Navy has recently found that selection variables or criterion used to predict 
undergraduate training success and early aviation training performance measures may 
not be related or may even be inversely related to performance in the advanced 
operational environment [Ref. 3]. 

One facility which has been invaluable in providing insight into the establishment 
of viable performance criterion for possible use in selection tests is the Navy Air 
Combat Maneuvering Range (ACMR). This facility allows multiple fighter aircraft to 
engage in a tactical environment, allowing the simulated employment of air to air 
weapons as a means of providing training in tactical skills, weapon utilization and 
weapon envelope recognition. Several problems arise from utilizing ACMR generated 
criterion, however. First, the high psychological stress levels associated with actual 
combat may be absent, secondly, individual performance is dependent on adversary 
performance. Thus an independent performance assessment is difficult if not 
impossible. Plans are ongoing to better utilize this facility in understanding aviator 
skills w r hich appear to enhance ACMR performance. 

Finally, Perceptual Psychomotor Testing and Division of Attention Tests are 
being improved by both the U.S. and Israeli Air Force respectively. Psychomotor 
testing has already made a unique contribution to the USAF UPT selection program. 
Israeli Air Force has likewise experienced success in predicting training success in high 
performance jet aircraft using the Division of Attention Test 2 [Ref. 6). 

We have now discussed what testing methods have been developed and exist 
today, and have established how important they are in developing the pilot selection 
program. We now suggest useful pilot selection methods for the ROKAF which 
hopefully will result in considerable resource savings and significant improvements in 
pilot quality. 



Approximately for 2,000 subjects, the data was significantly related to success in 
jet training. 



15 



III. AVIATOR SELECTION METHODS FOR ROKAF 



Although the human resources and materials for the flight training program in 
the ROKAF have increased only slightly, the number of pilot candidates is increasing 
every year as the number of Korean Air Force Academy (KAFA) cadets increases. 
Since the only admission requirement to the KAFA for flight training is a medical 
clearance, and most pass the medical examination, the ROKAF pilot training 
command cannot handle the number of cadets and candidates from other sources 
(AFROTC and Navy). This increased number of pilot candidates has created the 
necessity for minimizing the number of candidates lost due to poor or unsatisfactory 
proficiency in the UPT program; maximizing the probability that potential failures will 
be screened out prior to flight training. 

The present pilot selection procedures of the ROKAF do not adequately take 
into account candidates' mental abilities. The electrocadiogram, however, is being used 
for fighter pilots, and an abnormal electrocadiogram will prohibit continuation of flight 
training. All of the KAFA cadets who pass the medical examination enter the flight 
training program and have a chance to fly. Flight training starts early before 
graduation from the KAFA and ends about a year after graduation. This system, with 
flight training begining early before graduation was adapted during the 1980's. 
However the current system does not appear to represent a significant improvement in 
previous program. Cadets who fail as a cadet fmd it difficult to continue the program 
and those who fail as an officer after graduation and commissioning fmd it difficult to 
adapt to different specialties. Candidates are divided into several subgroups in order to 
meet the limited training squadron resources. Graduates of the subgroups enter the 
next phase of training with attrites returning to the KAFA. Attritions continue 
through the flight training - from the primary training program up to the advanced 
program over several years. The only way that candidates could be differentiated is on 
some amount of actual flying at the training squadron and the failures would be totally 
subjective based on instructors' feeling, knowledge and their experiences, except for 
voluntary withdrawal or drop on request (DOR). 

The following demonstrates a useful way to help in solving the existing short 
comings of the current ROKAF pilot training program and suggests pilot candidate 
selection methods which might be used as a starting point in alleviating these problems. 
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A. METHODS 



This section suggests selection methods for the ROKAF which are currently 
being used in the U.S. The purpose of the tests are to select candidates who have a 
high probability of completing the flight training program. Multitask performance is 
being used in USN and it has enjoyed considerable success in predicting flight training 
[Ref. 7], The USAF uses a psychomotor screening system for selecting candidates as 
well as identifying whether the candidate is suited for fighter pilot training [Ref. 8]. 
The USAF also suggests that an integrated model including psychomotor testing and 
other available information - AFOQT scores, technical degree, prior flying experience, 
age of subjects, etc. - is better than using only the psychomotor test. All three 
methods use a psychomotor test since the psychomotor scores have been found to 
relate significantly to flight training pass/fail criteria. As the USN adapted the EEG 
recordings as a direct result of the Danish Air Force as w r e discussed in Chapter II, the 
ROKAF may choose one of these three methods or a combination and apply it to 
ROKAF cadet selection with some possible modifications. The following text describes 
the three methods which could be used and modified for the ROKAF and shows how 
the predicted results may be related to the actual outcomes. 

1. Multitask Performance 

The USN had great success in predicting student success in primary flight 
training utilizing multitask performance tests in 1986 [Ref. 7]. A Psychomotor Test 
(PMT) and a Dichotic Listening Task (DLT) w'ere administered under single-task and 
multitask conditions separately and in combination. Testing procedures were as 
follows: [Ref. 7] 

1) Psychomotor Test (PMT) 

• The PMT requires subjects to simultaneously center tw r o visually 
presented cursors on fixed targets displayed on a CRT screen. The test 
performance measurements were the accumulated X, Y, and Z axis error 
distances between the target and the cursor point. A diagram of the 
experimental apparatus of Psychomotor Task (PMT) is depicted in Figure 
3.1 . 

2) Dichotic Listening Task (DLT) 

• The DLT first requires subjects to attend to auditory information 
presented to one ear, while ignoring similar information presented to the 
opposite ear, and then after considering an auditory cue, to switch 
attention rapidly to the previously attended ear, or maintain attention to 
the previously attended channel. The modified Dichotic Listening task 
(DLT) [Ref. 9] consisted of letter-digit string sets of 24 listening trials for 
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Figure 3.1 Experimental Apparatus (adapted from Ref.7). 

each of 3 counter-balanced response modalities. Written, keypad and 
verbal response conditions were employed. The subjects were instructed 
to direct attention to one ear while ignoring the other, and correctly 
report the nine digits per trial. An example DLT trial is depicted in 
Figure 3.2 . The single-task performance measure for the DLT was 
simply the number of correct responses per 24 trials. 

3) Multitask Performance 

• For the multitask condition, subjects performed two sessions of the DLT 
and PMT simultaneously. Performance measures for the PMT and DLT 
in the multitask conditions were identical to those of the single-task 
conditions except that multitask DLT performance was based on 12 
rather than 24 trials. 

The tests were performed in order such as: 

1) Psychomotor Test (PMT), 

2) Dichotic Listening Task (DLT) and 
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PART I 



Left Ear R8NSMY2GB7FL6RL5 

"Right" (Vocal Channel "attend" Command) 

Right Ear YL3SR4FZ9XF0FN1L 

PART II 

Left Ear B F 4 3 7 9 

"Left" (Vocal Channel "attend" Command) 

Right Ear G L 1 5 6 2 



Figure 3.2 DLT Trial Examples (adapted from Ref.7). 

3) Multitask, that is, PMT and DLT simultaneously. 

To validate the test criteria, subjects' flight performance were tracked until 
completing pilot training. Table 2 shows the mean and pass/fail correlation of each 
test measure. Correlations between each testing measure are presented in Table 3 . As 
you can see in Table 2 , testing measures 1, 5, 9, and 11 are significantly related to the 
flight training pass/fail criteria. Table 3 presents a correlation matrix of test measures. 

2. Psychomotor Test 

The USAF has utilized psychomotor testing in relation to pilot selection. The 
USAF primary objective has been an attempt to develop reliable and valid test devices 
and validation of test scores. In 1969, the Air Force Human Resource Laboratory 
(AFHRL) attempted to determine the reliability of two computer-based psychomotor 
tests (i.e., Two-Hand Coordination and Complex Coordination) [Ref. 8J. Both tests 
were employed to measure the subject's ability to coordinate movements. Subjects 
were from Air Force Academy (AFA), Air Force Reserve Officer Training Corps 
(AFROTC), and Officer Training School (OTS). The largest group of the sample was 
AFROTC and additional subjects from Williams AFB were used for validation of 
testing results. 
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TABLE 2 



DESCRIPTIVE STATISTICS FOR SINGLE AND MULTITASK 
MEASURES AND SELECTION TESTS 



(percent correct) 



Test Measure 


Mean 


Pass; Fail 
(r) 


1. PMT Session 1, X,Y,Z error combined 


19116.36 


-.283 


2. PMT Session 2, X,Y,Z error combined 


11541.20 


-.028 


3. PMT Session 3, X,Y,Z error combined 


9896.90 


-.241 


4. PMT Session 4, X,Y,Z error combined 


9550.82 


-.120 


5. DLT written response. No. correct 


211.48 (97.9%) 


.360 


6. DLT keypad response, No. correct 


210.60 (97.5%) 


.162 


7. DLT vocal response, No. correct 


207.52 (96.1%) 


.244 


8. PMT X,Y,Z error (with keypad DLT) 


21816.92 


.036 


9. DLT keypad response (with PMT) 


102.34 (94.8%) 


.413 


10. PMT X,Y,Z error (with vocal DLT) 


14038.10 


-.002 


11. DLT vocal response (with PMT) 


101.44 (93.9%) 


.395 


12. AQT standard score 


5.66 


.172 


13. FAR standard score 


6.80 


.361 



** Measure 1-7 : single-task 

** Measure 8-11 : multitask 



** Measure 12,13 : selection test 



(From Reference 7) 



Testing procedures were as follows: [Ref. 8] 

1) Two-Hand Coordination 

• The first of the tests presented a triangular-shaped target and a cross- 
shaped pipper on the CRT (Figure 3.3). The computer moved the target 
in an elliptical path and with varying speeds - faster near the 4 o'clock 
and slower near the 11 o'clock position. The subjects were instructed to 
use both joysticks simultaneously (one is for up-down and the other is for 
left-right) in a coordinated manner to move the pipper, keeping it as close 
as possible to the target on the CRT. The computer measured absolute 
distance from the pipper to the target. The horizontal error (Xj) and 
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TABLE 3 



CORRELATION MATRIX FOR SINGLE AND MULTITASK 
MEASURES AND SELECTION TESTS 



Measure 1 
1 


2 


3 


4 


5 


6 


7 


8 


9 


10 


1 

2 


.401 




















3 


.727 


.544 


















4 


.520 


.667 


.838 
















5 


-.626 


-.131 


-.694 


-.443 














6 


-.616 


-.490 


-.523 


-.453 


.540 












7 


-.494 


-.182 


-.407 


-.325 


.708 


.544 










8 


.393 


.500 


.501 


.597 


-.385 


-.503 


-.612 








9 


-.708 


-.244 


-.782 


-.508 


.869 


.588 


.563 


-.348 






10 


.526 


.298 


.532 


.443 


-.531 


-.468 


-.669 


.873 


-.445 




11 


-.737 


-.243 


-.732 


-.457 


.913 


.540 


.644 


-.363 


.887 


-.529 


12 


-.157 


-.067 


.060 


-.021 


.058 


.104 


.147 


.048 


.116 


.037 


13 


-.368 


-.239 


-.223 


-.287 


.050 


.144 


.223 


.254 


.144 


.230 



** See Table 2 for description of numbered variables. 
(From Reference 7) 



vertical error (Yj) measurement were accumulated during 5-min test 
period. 

2) Complex Coordination 

• The second test presented a set of cross-hairs centered on the CRT, a dot- 
shaped pipper, and a thin vertical bar at the bottom of the CRT (Figure 
3.4). The subject controlled the pipper, both horizontally and vertically, 
using the floor-mounted joystick. The control responses were the reverse 
of what is traditionally required on aircraft to reduce the advantage of a 
subject with prior flying experience. The subjects were instructed to use 
the joystick to center the pipper horizontally and vertically on the 
intersection of the cross-hairs and simultaneously press the appropriate 
rudder-style pedal to center the "rudder bar" over the lower part of the 
vertical cross-hair. The computer measured in inches and accumulated 
during 5-min test period the the absolute distance from the pipper to the 
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PSYCHOMOTOR TEST I 



LEFT — RIGHT HAND COORDINATION 
(10 minutes) 






Figure 3.3 Two-Hand Coordination Test Depiction (adapted from R.ef.7). 

intersection of the cross-hairs and from the rudder bar to the vertical 
cross-hair. The three scores obtained from this test were the cumulative 
horizontal error (X 2 ), vertical error (Y' 2 ), and horizontal error for rudder 
bar (Z 2 ). 

Table 4 presents five error scores (Xj, Yp X 2 , Y 2 . Zj) for each subject. 
These data show that the Two-Hand Coordination test was more difficult than the 
Complex Coordination. There might also be some common information about the 
subjects since all five test scores measured psychomotor ability. Table 5 shows 
intercorrelations across the tests. Two psychomotor tests were measuring different 
abilities with very little in common between the tests as indicated in Table 5 . To 
validate the psychomotor testing scores, the mean scores of UPT graduates and 
eliminees were compared following completion of flight training. Table 6 presents the 
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PSYCHOMOTOR TEST II 



STICK & RUDDER SKILLS 
(10 minutes) 





A ^ 




V X 






1 X' 


1 ^ 





PIPPER 
RUDDER BAR 




Figure 3.4 Complex Coordination Test Depiction (adapted from Ref.7). 

mean of the psychomotor scores among three categories: UPT graduates, all UPT 
eliminees, and UPT flying deficiency eliminees. All five scores had significant 
differences between graduates and either category of eliminees. Note that for all three 
of the Complex scores (X 2 , Y 2 , and Z 2 ) were significantly worse than the graduates' 
means. These results were used to identify subjects more likely to be fighter pilots. 
Figure 3.5 presents this result of all five means for both fighter-recommended (FR) 
and non-FR students. The FR students had significantly better scores than the non- 
FR students. The psychomotor screening equation was finally developed and the most 
useful screening score based on psychomotor ability can be computed from the 
following equation (Ref. 8]: 
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TABLE 4 

DESCRIPTIVE STATISTICS 



Test Score Mean 

Two-Hand Coordination 

XI (Horizon) 14,709 

Y1 (Vertical) 16,748 

Complex Coordination 

X2 (Horizon) 4,514 

Y2 (Vertical) 3,860 

Z2 (Rudder) 5,580 

(From Reference 8) 



(inch) 



SD 


Range 


5,400 


428 - 65,268 


5,538 


5,687 - 57,292 


6,520 


326 - 71,040 


6,445 


486 - 76,320 


6,018 


167 - 71,040 



TABLE 5 

CORRELATION MATRIX 





X 1 


Y 1 


X 2 


XI (Horizon) 








Y1 (Vertical) 


.87 






X2 (Horizon) 


.20 


.25 




Y2 (Vertical) 


.19 


.23 


.86 


Z2 (Rudder) 


.20 


.24 


.77 



(From Reference 8) 



Yj = a + (b, x xij) + (b 2 x Y2 j) 

where Yj = psychomotor screening score of /th subject 
a = constant (0.95648) 
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TABLE 6 

MEAN PSYCHOMOTOR SCORES BY UPT OUTCOME 






X 1 


Y 1 


X 2 


Y 2 


Z 2 


Graduate Means (N = 1,348) 


14,315 


16,341 


3,559 


2,058 


4,725 


All Elim. Means (N = 377) 


15,829 


17,621 


4,936 


4,173 


6,678 


Flying Deficiency (FD) 


16,302 


18,007 


5,593 


4,702 


7,580 


Elim. Means (N = 199) 
(From Reference 8) 



bj = weight of psychomotor XI score (-.0000075748) 
b 2 = weight of psychomotor Y2 score (-.000020375) 

XI- = psychomotor Xj score of /th subject 
Y2j = psychomotor Y 2 score of /th subject 

This equation provides a screening score based on psychomotor ability. These scores 
could be used in a selection system by rank ordering; and the best candidates could be 
selected to fill the required training quotas. 

3. Integrated Selection System 

Although psychomotor testing scores are significantly related to pass/fail 
criteria in UPT, the ideal pilot selection method would consider all available 
informations on the subjects. 

Current operational USAF pilot selection criteria consider information from 
the five composite scores of the AFOQT, age at the start of UPT, possession of a 
college technical degree, and possession of a private pilot's license or completion of a 
USAF light aircraft flying program [Ref. 8]. Table 7 lists the correlations of all 
available measures with the UPT pass/fail criterion for all reasons (FY79 to FY83). 

The integrated model was developed based on the USAF psychomotor testing. 
Because pilot candidates are chosen differently, depending on their source of 
commission, three different Integrated Pilot Candidate Selection Models (IPCSMs) 
were developed. 

The three IPCSMs equations were developed and will produce a score 
corresponding to the probability of UPT success using the following equations : 
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MEAN PSYCHOMOTOR SCORE 





in 

UJ 




o 

o 

o 

CD 



o 

o 

o 

04 



J L 

o 

o 

o 

a: 




o 



Figure 3.5 Mean Psychomotor Scores by FR/non-FR. 
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TABLE 7 

CORRELATIONS OF IPCSM PREDICTIONS WITH LPT OUTCOME 


Predictors Correlation with LPT 


Sample Sizes 


FSP Pass/ Fail 


Pass/ Fail 
.137 


1,534 


FSP Final Grade 


.271 


1,534 


FSP Sum of Grades 


.303 


1,534 


AFOQT Pilot Score 


.158 


4,460 


AFOQT Navigator Tech Score 


.148 


4,460 


AFOQT Academic Score 


.080 


4,577 


AFOQT Verbal Score 


.007 


4,576 


AFOQT Quantitative Score 


.138 


4,577 


Psychomotor XI Score 


-.118 


1,918 


Psychomotor Yl Score 


-.099 


1,919 


Psychomotor X2 Score 


-.153 


1,906 


Psychomotor Y2 Score 


-.181 


1,910 


Psychomotor Z2 Score 


-.146 


1,916 


Age 


-.120 


8,438 


Gender 


.033 


8,438 


Race 


.110 


8,292 


Possession of Tech Degree 


.111 


8,183 


(From Reference 8) 







1) IPCSM I (OTS Model) 



Yi = a + (bj x X 1 -) + (b 2 x Y2i ) + (b 3 x age^ 
+ (b 4 x AFOQTP^ + (b 5 x AFOQTNj) 
+ (b 6 x FSPlj) + (b 7 x FSP2j) 



where 

a 



b 



l 



= IPCSM I score for z'th subject 
= constant (0.10192) 

= weight for psychomotor XI score (-.000015211) 
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u 2 

b 3 

b 4 

b 5 

b 6 

b 7 

Xli 

Y2 i 

agej 

AFOQTPj 

AFOQTNj 

FSPlj 

FSP2j 



weight for psychomotor Y2 score (-.000010618) 

weight for age of subject (-.0012) 

weight for AFOQTP/AFOQT pilot score (0.00112) 

weight for AFOQTN/AFOQT navigator score (0.00029067) 

weight for FSP1/FSP final grade (0.08249) 

weight for FSP2/FSP sum of grades (0.01923) 

psychomotor XI score of /th subject 

psychomotor Y2 score of /th subject 

age at start of UPT of/th subject 

AFOQT pilot composite score of /th subject 

AFOQT navigator composite score of /th subject 

FSP final grade of /th subject 

FSP sum of grade of /th subject 



2) IPCSM II (AFROTC Model) 



Yj = a + (b 1 x Xlj) + (b 2 x Y2j) + (b 3 x agej) 
+ (b 4 x AFOQTPj) + (b 5 x AFOQTNj) 



where 



Y i 

a 

b, 



= IPCSM II score for /th subject 
= constant (1.72485) 

= weight for psychomotor XI score (-.0000030409) 

= weight for psychomotor Y2 score (-.000022526) 

= weight for age of subject (-.04517) 

= weight for AFOQTP/AFOQT pilot score (0.00124) 

= weight for AFOQTN/AFOQT navigator score (0.00225) 



3) IPCSM III (AFA Model) 



Yj = a + (b t x xij) + (b 2 x Y2 j) + (b 3 x ag ej) + (b 4 x techj) 



where Yj 
a 
b 



l 

b 2 

b 3 

b 4 

techj 



IPCSM III score of /th subject 
constant (1.43056) 

weight for psychomotor XI score (-.0000050818) 
weight for psychomotor Y2 score (-.0000072309) 
weight for age of subject (-.02111) 
weight for possession of technical degree (.08756) 
1 if /th subject has a technical degree, 0 otherwise 
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MEAN IPCSM SCORE 



m GRADUATES 
□ ELIMINEES 




CORRELATION = .456 



.297 .205 



Figure 3.6 Mean IPCSM Scores by LPT outcome. 
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The mean of the IPCSM scores are graphically presented in Figure 3.6 . The 
differences between the means were significant and generated the better prediction than 
using the psychomotor test only. One way to select candidates is to accept only those 
candidates who obtain a screening score above a predetermined cutoff. The cutoff 
score would be determined by the quota required and attrition rate. The higher the 
cutoff score is set, the greater the impact on attrition in pilot training program. The 
FY83 UPT sample was used to cross-validate the three IPCSMs and resulted in the 
IPCSM scores were significantly correlated with UPT outcome. Since the "Whole 
Person Concept" is imbedded in this multi-dimensional IPCSM approach to pilot 
selection, the integrated selection method has better predictors than the psychomotor 
test only does. 

As we can see through the Chapter, all three pilot selection methods include a 
psychomotor testing which means psychomotor testing scores are significantly different 
between the successes and the failures. It should have the same result if psychomotor 
tests were applied to ROKAF. The only thing different between U.S. and ROKAF 
would be the cutoff score as a different required quota. The ROKAF decision maker, 
however, must know the advantages of using an experimental testing method in terms 
of their utilities or cost effectiveness. Chapter IV provides one way to make a decision 
and evaluate the outcomes. 
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IV. EVALUATION OF TESTING METHOD 



This chapter discusses how to make appropriate decisions which maximize an 
overall Measure of Effectiveness (MOE) or Utility which might be used in a pilot 
selection program. The final goal of personnel selection programs has traditionally 
been to achieve maximum accuracy of measurement and efficiency of prediction. 
Decision theory recognizes that the outcomes of predictions are of primary importance 
to individuals and organizations. As discussed in Chapter III, for example, Korean 
AFA cadets are commissioned as officers after four years of attendance. Some fail 
either in undergraduate or operational flight training. This results in high attrition 
cost, dissatisfaction, lowered morale, and a number of officers totally unneeded in the 
service. Since the measurement and prediction are simply technical components of a 
system designed to make decisions, we have to defme the MOE before making any 
decisions. There are, however, a number of possible MOEs in most decision problems. 
For example, there would be three possible MOEs in a pilot selection problem: 

1) Costs - selection, attrition and training, 

2) Pilot quality, and 

3) Probability that the quota will be satisfied. 

Thus decision outcomes should be evaluated in terms of their consequences for 
individuals and organizations. That is, in terms of their overall MOE or Utility. This 
provides us with a consistent way in which to combine different figures of merit or 
subjective and judgemental factors, and probabilistic aspects of outcomes into an 
overall MOE. 

To construct and evaluate a decision model, we have to find parameters which 
function in pilot selection outcomes. The major factors are [Ref. 10]: 

a) the base rate (the proportion of persons currently accepted who are 
satisfactory) 

b) the selection ratio (the proportion of applicants accepted) and 

c) the validity coefficient (the predictor criterion correlation). 

Taylor and Russell [Ref. 10] published a series of tables which illustrated that the 
value of a test was a function of the three considerations listed above. 
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A. BASE RATE (BR) 



t' 




Figure 4.1 Four possible outcomes of personnel selection (adapted from Ref. 11). 

The possible outcomes of personnel selection programs are illustrated in Figure 
4.1 . This figure depicts a bivariate plot for a selection test with a validity of .50. The 
horizontal axis, BB', represents a particular base rate of satisfactory criterion 
performance, in this case, 50 %. The vertical axis, CC\ denotes the cutting score for a 
50 % selection ratio. Thus, Quadrant A contains a number of satisfactory subjects 
above the cutting score (valid positives, VP), Quadrant B contains unsatisfactory 
subjects above the cutting score (false positives, FP), Quadrant C contains 
unsatisfactory subjects below the cutting score (valid negatives, VN), and Quadrant D 
contains satisfactory subjects below the cutting score (false negatives, FN). [Ref. 11] 
Once the cutoff score is set, a definite number of predictions of success and failure are 
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made. As Figure 4.1 shows, the total number of .predictive positives, N pp , may he 
calculated from : 



Npp = VP + FP 



Similarly, the total number of predictive negatives, N_ n , is given by: 



N 



pn 



The total number of actual positives, N 



= FN + VN 
ap , is given by: 



N„ n = FN + VP 
ap 

And the total number of actual negatives, N T an , is calculated by: 

N an = VN + FP 

The probability of a valid positives, P(VP), is thus: 



P(VP) = VP/(N ap + N an ) = VP/TN 

where TN is the total number of candidates. Figure 4.2 presents the probabilities and 
their relationships. 

When outcomes are expressed as probabilities they are referred to as "hit rates". 
The probabilities or proportions of actual positives that exist in the total sample of 
candidates is called the "Base Rate" (BR). 3 The probability may be calculated directly 
BR = N' /TN or from the probabilities of two outcomes BR = P(FN) + P(VP) as in 
Figure 4.2 . The BR has reference to the base rate of the positive class and so the 
probability or base rate of negative in the total sample is given : 1 — BR. 

Selection measures are most useful when BRs are about .50 [Ref. 10]. As the 
BR departs radically in either direction from .50, the benefit of an additional predictor 
becomes questionable. 



3 The proportion of candidates who would be successful in flight training without 
use of the test. 
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Actual Diagnosis 


Test Diagnosis 


Total 


Positive Negative 


Positive 


P(VP) = VP/TN P( FN) = FN/TN 

valid positive rate false negative rate 

(Proportion of (Proportion of 

positives called positives called 

positive) negative) 


BR = N ap /TN 
Base Rate 
(Probability of 
actual positives) 




Negative 


P(FP) = FP/TN P(VN’) = VN/TN 

false positive rate valid negative rate 

(Proportion of (Proportion of 

negatives called negatives called 

positive) negative) 


1 - BR = 

N /TN 
lN an' 1 • 

(Probability of 
actual negatives) 


Total 


SR=Npp/(Npp + Np n ) 1 - SR = 
Selection Ratio N p n /( N pp + N pn ) 

(Probabihty of (Probability of 

predicted positives) predicted negatives) 





Figure 4.2 Outcomes Expressed as Probabilities. 

B. SELECTION RATIO (SR) 

The probability or proportion of predicted positives among the total sample of 
candidates is called the "Selection Ratio" (SR). 4 This probability may be calculated 
directly as follows: 



SR 



\ '(N + N ) 

• pp ,v ‘ pp ‘ pn' 



4 The proportion of candidates selected with the use of test. 
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or from the probabilities of two outcomes such that . 



SR = P(VP) + P(FP) 



Whenever a quota exists on the total number of candidates that may be 
accepted, the selection rate becomes more important. As the SR approaches 1.0, for 
example, which means all candidates must be selected, it will be dominated by other 
alternatives and as the SR approaches zero, it dominates all other alternatives which 
means a decision maker can afford to be selective. Cronbach and Glesser [Ref. 12] 
have shown that when the frequency distributions of the two groups (accept/reject) are 
plotted separately along the same base line, the optimum cutting score for 
distinguishing between the two groups will occur at that point where the two 
distributions intersect. However, in most practical selection situations, the decision 
maker is not free to vary selection ratios. 

C. VALIDITY COEFFICIENT 

Evaluating the benefit obtained from tests is of considerable practical interest and 
has been a problem in industrial psychology. The relation of benefit to validity has 
long been regarded as an important question. Most attempts to evaluate benefits have 
focused on the validity coefficient. As at least five formulas for interpreting the validity 
coefficient have been given, we face the problem of choosing between them or of 
modifying them. [Ref. 12] 

The validity coefficient is usually defined as the correlation of test score with 
outcome or criterion score. We shall use r YV in our discussion because the decision 
maker is interested in predicting the evaluated outcome, that is, the outcome expressed 
in utility units. The rule having the longest history translates the validity coefficient 
r X y into the "index of forecasting efficiency" [Ref. 12], 



Although the validity coefficient r YV is a useful index, there are many ways in 

Ay 

which it falls short of accurately predicting success. From a practical point of view, the 
number of correct decisions made by a psychological test or assessment is more 
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important than the degree of association which exists between predicted and obtained 
scores [Ref. 12]. When two variables are expressed in dichotomous form (success- 
failure, select-reject) as in Figure 4.1 , the correlation between them is given by the 
point-biserial "r p coefficient". A simplified computational formula for the r p coefficient 
is: [Ref. 13] 



P(VP) - BR x SR 

f P [ BR(1 - BR)SR( 1 - SR) ] 1/2 

The value of r p is computed knowing only three values: 

1) The probability of a valid positive, P(VP), 

2) The base rate, BR, and 

3) The selection ratio, SR. 

The computational formula for the r p coefficient can be solved for the probability 
of valid positives: 



P(VP) = BR x SR + r ^BRO - BR)SR(1 - SR) 



This expression is especially useful in evaluating the outcomes of test prediction. 
Given the validity of a test r p , the base rate, BR, and the proportion of candidates to 
be selected, SR, it is possible to completely specify the outcomes of predictions. 

Taylor and Russell [Ref. 14] denounced the practice of evaluating the 
effectiveness of predictor variables solely on the basis of their validity coefficients. 
They prepared tables for determining the effectiveness of a selection test by estimating 
the proportion of selectees that are considered satisfactory when the test validity, base 
rate, and selection ratio are known or assumed. 

D. DECISION STRATEGIES 

There are mainly two different decision strategies which are single-stage 
(nonsequential) and multi-stage (sequential) [Ref. 12]. The major difference between 
these two strategies is when the terminal decision is made. The general decision 
process is illustrated schematically in Figure 4.3 . 
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Outcome 



Figure 4.3 Schematic View of a Decision Process (adapted from Ref. 12). 

1. Single-Stage Strategy 

A test or test battery is employed to all candidates. Based on a predetermined 
cutting score, the terminal decision is made. Individuals are either selected or rejected 
and no further decisions are made. This strategy corresponds to selecting candidates 
randomly or by a medical examination only in pilot selection. 

2. Multi-Stage Strategy 

Terminal decisions are made only from the view point of a decision maker in 
multi-stage strategy. The cutting score on a test battery may be used to make 
investigatory decisions. This decision calls for additional information, dictating what 
test or procedure wall be used to gather that information. This then leads to a further 
decision. The investigatory decisions may continue through a number of stages of 
subsequent testing until terminal decisions are made regarding all applicants. This type 
of decision process is used to select pilot candidates using one or more experimental 
testing method until a decision maker is satisfying. 
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E. THE EXPECTED UTILITIES (EU) 

As discussed at the beginning of this Chapter, utility provides a consistent way to 
compare each different selection method. The comparison would be accomplished by 
expected utility, i.e., maximizes the expected utility across the possible outcomes. If 
the four possible outcomes such that valid positive, false positive, valid negative and 
false negative have utility U V p , Ufp , U yn and Uf n respectively, then the expected 
utility can be described generally: 

EU = U vp x P(VP) + U fp x P(FP) + U vn x P(VN) + U fn x P(FN) - Uy 

where Uy is the utility of testing method employed. Under different decision strategies, 
the expected utility should be considered separately so that we can figure out which 
decision strategy is better with which testing method such that expected utility for all 
possible outcomes can be maximized. 

A number of different decision strategies can result from combining different 
testing procedures. In the ROKAF selection problem, consideration in given to two 
simple decision strategies, i.e., single-stage decision with random or medical 
examination, or double-stage decision with medical and one experimental test. This is 
based on the assumption that ROKAF has only two alternatives. 

1. Single-Stage Decision with Random Selection 

If the Korean Air Force uses a random selection process, providing that the 
SR, BR, number of applicants N, then the expected utility is going to be: 

EU = U vp x P(VP) + U fp x P(FP) + U vn x P (VN) + U fn x P(FN) 

where P(VP) = BR x SR since r p = 0 
P(FP) = SR - P(VP) 

P(FN) = BR - P(VP) 

P(VN) = (l - BR) - P(FP) = (1 - SR) - P(FN) and 
Uy _ 0 since no testing employed. 

Whenever the ROKAF has data available this calculation gives the ROKAF a 
consistent way to compare alternatives with their utilities. For the present effort, it is 
possible to simulate ROKAF data and evaluate the results. Suppose ROKAF data is 
as follows: 
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• SR = .6, BR = .5, N = 500, 

• utility of valid positive, in dollar, L)' = S80,000, 

• utility of false positive, Ujp = - S80,000, 

• utility of valid negative, U yn = S50,000, 

• utility of false negative, Uj~ n = 0, 

Now we can compute the expected utility of a random selection process when 
the ROKAF uses random selection of KAFA cadets; 

P(VP) = BR x SR = .5 x .6 = .3 

P(FP) = SR - P(VP) = .6 - .3 = .3 

P(FN) = BR - P(VP) = .5 - .3 = .2 

P(VN) = (1 - SR) - P(FN) = (1 - .6) - .2 = .2 



then 



EU = 80,000 x .3 - 80,000 x .3 + 50,000 x .2 + 0 x .2 
= S 10,000 per candidate 



2. Single-Stage Decision with Medical Examination 

When a medical examination is employed the probability of valid positive 

becomes: 



P(VP) = PS x SR 

where PS is proportion of satisfactory candidates among those selected by a test. 
Taylor and Russell developed a table for this PS value and Appendix C provides these 
values as different BR and SR which is similar to Taylor and Russell tables but more 
appropriate for point-biserial correlation, r , by Abrahams, Alf and Wolfe [Ref. 10]. If 
r p is .25, then the PS = .582 with SR = .6, BR = .5 at Table given in Appendix C. 

Assuming the utility of medical examination is S50 then the expected utility 
becomes: 

P(VP) = PS x SR = .582 x .6 = .3492 
P(FP) = SR - P(VP) = .6 - .3492 = .2508 
P(FN) = BR - P(VP) = .5 - .3492 = .1746 
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P(VN) = (1 - SR) - P(FN) = (l - .6) -..1746 = .2254 



thus, 



EU = 80,000 x .3492 - 80,000 x .2508 + 50,000 x .2254 - 50 
= S 19,092 per candidate 

which is more than the previous random selection with the amount of S9,092 utility 
using only a medical examination. This result indicates that selection via the medical 
examination is more effective than random selection. In other words, the medical 
examination increases a certain amount of utility of possible personnel selection 
outcomes. 

3. Double-Stage Decision with Medical and an Experimental Test 

Since the terminal decision is made via medical and experimental test 

decisions, it is reasonable to select enough candidates at the first decision so as to meet 

the required quota at the second decision. In the second decision, the terminal decision 

is made concerning the provisionally accepted applicants on the basis of the first test. 

If we assume that SR = .7 in this case and r = .8 from the table with the same BR 

P 

then the value of PS equals .709. With the utility of the experimental test equal to S80, 
the calculation of the expected utility is, therefore, as follows: 

P(VP) = PS x SR = .709 x .7 = .4963 

P(FP) = SR - P(VP) = .7 - .4963 = .2037 

P(FN) = BR - P(VP) = .5 - .4963 = .0037 

P(VN) = (1 - SR) - P(FN) = (1 - .7) - .0037 = .2963 



then 



EU = 80,000 x .4963 - 80,000 x .2037 + 50,000 x .2963 - 50 - 80 
= 538,093 per candidate 

which is more than three times the random selection utility and more than 519,000 
utility per candidate via medical test only. This result shows that if the ROKAF 
changes the present pilot selection method and uses an experimental test method with 
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medical examination, the increased expected utility of that change is about S 19,000 per 
candidate even if using small scale simulated data. 

In other words, the potential advantages in changing the present ROKAF 
pilot selection system are obviously considerable. 

F. CAPER MODEL EVALUATION 

To this point decision strategies have been discussed in terms of their utilities and 
advantages. Managers of personnel systems, however, justifiablely demand an estimate 
of the payoff, in dollars, which can be expected to result from the implementation of a 
proposed selection program. It might appear that, for a given test validity and base 
rate, a personnel manager always should lower the selection ratio (i.e, become more 
selective) this strategy may or may not be cost effective since lowering selection ratio 
causes expanding recruit and selection efforts to meet the required quota. 

The purpose of this section is again to evaluate whether the changes are still 
feasible in terms of cost-effectiveness by using the Cost of Attaining Personnel 
Requirements (CAPER) model. Sands [Ref. 11] defined the CAPER model as follows: 

• "The CAPER model is designed to evaluate the cost consequences of alternative 
recruiting-selection strategies. Specifically, the CAPER model determines an 
optimal recruiting-selection strategy for minimizing the estimated total cost of 
recruiting, selecting, inducting, and training a sufficient number of persons to 
meet a specified quota of satisfactory personnel. In addition, the CAPER 
model considers the cost of an erroneous acceptance (selecting a person for a 
training program who subsequently fails to graduate) and the cost of erroneous 
rejection (rejecting a person who would have succeeded if given the 
opportunity)." 

Suppose that the ROKAF General Staff provides specific information for pilot 
selection policy such that the quota, the base rate and cost data for selection, 
recruiting, induction, training, erroneous acceptance and rejections, etc. Example of 
input data for the ROKAF is provided in Table 8 [Ref. 13] and shows that 500 of 1,000 
persons graduated, indicating a .500 base rate. Suppose the data was given as follows: 

• the cost of recruiting an individual is S4,000, 

• the cost of the present selection procedure (medical examination) is SI 50 
including the physicians' salaries and laboratory fees, 

• the cost of administering and scoring the experimental test is S100, 

• the induction cost per individual is estimated as S150, 

• the cost for training is S3, 500, 

• the cost of erroneous acceptance is set at SI, 000, and 

• the cost of erroneous rejection is estimated as S800. 
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TABLE 8 _ 

EXAMPLE CAPER MODEL INPUT DATA FOR ROKAF 



Cuting 

Score 


Graduates 


Failures 


Qualified for Acceptance 


Qualified for Acceptance 


Frequency 


Number 


Proportion 


Frequency 


Number 


Proportion 


0 


0 


500 


1.000 


0 


500 


1.000 


1 


0 


500 


1.000 


7 


500 


1.000 


2 


0 


500 


1.000 


10 


493 


.986 


3 


0 


500 


1.000 


17 


483 


.966 


4 


2 


500 


1.000 


29 


466 


.932 


5 


6 


498 


.996 


45 


437 


.874 


6 


11 


492 


.984 


58 


392 


.784 


7 


21 


481 


.962 


68 


334 


.668 


8 


36 


460 


.920 


72 


266 


.532 


9 


52 


424 


.848 


66 


194 


.388 


10 


66 


372 


.744 


52 


128 


.256 


11 


72 


306 


.612 


36 


76 


.152 


12 


68 


234 


.468 


21 


40 


.080 


13 


58 


166 


.332 


11 


19 


.038 


14 


45 


108 


.216 


6 


8 


.016 


15 


29 


63 


.126 


2 


2 


.004 


16 


17 


34 


.068 


0 


0 


.000 


17 


10 


17 


.034 


0 


0 


.000 


18 


7 


7 


.014 


0 


0 


.000 
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Given the simulated data, the ROKAF can plug into the CAPER model the 
equations and compute costs the present ROKAF selection process and the new 
selection process. The set of equations is used once for each possible cutting score on 
the new selection process. A cutting score of eight on the test will be used for 
illustration. 

Equation (1) gives the formula for estimating the number of applicants who must 
be recruited in order to meet the quota : 

NR = Q/ [ (BRXPGj) ] BR > 0, PG; > 0 (1) 

where NR is the number recruited, 

Q is the quota of satisfactory personnel, 

BR is the base rate, and 

PG ; is the proportion of graduates who would qualify for acceptance at the /th 
cutting score on the test. 

Substituting the data of the present ROKAF selection procedure into Equation 
(1) gives: 



NR p = 50/ [(0.50)(1.00)] = 100 
Similarly, for the experimental selection procedure ( /= 8): 

NR e = 50/ [ (0.50X0.92) ] = 109 

Equation (2) gives the formula for estimating the number of erroneous 
acceptances : 



NEA = NR(1 - BR)PF. (2) 

where NEA is the number of erroneous acceptances, 

PFj is the proportion of failures who would qualify for acceptance at the /th 
cutting score on the test, and the remaining symbols are defined above. 

Substituting the data of the present ROKAF selection procedure into Equation 
(2) gives: 
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NEA = (100)(1 - 0.50)(1.00) = 50 
Similarly, for the experimental selection procedure ( /= 8): 



NEA e = (109X1 “ 0.50X0.532) - 29 



Equation (3) gives the formula for estimating the number of erroneous rejections: 

NER = (NR)(BR)(1 - PG. ) (3) 

where NER is the number of erroneous rejections, and the remaining symbols are 
defined above. 

Substituting the data of the present ROKAF selection procedure into Equation 
(3) gives: 



NER = (100X0.50)(1 “ 1-00) = 0 
Similarly, for the experimental selection procedure ( /= 8): 

NER e = (109)(0.50X1 “ 0.92) = 4 

Equation (4) gives the formula for estimating the number of persons who will 
accepted : 



NA = Q + NEA (4) 

where NA is the number accepted and the others are the same as above. 

Substituting the data of the present ROKAF selection procedure into Equation 
(4) gives: 



NA p = 50 + 50 = 100 

Similarly, for the experimental selection procedure ( /= 8): 
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NA e = 50 + 29 = 79 

The total cost of the present selection procedure and the experimental selection 
procedure should be calculated by separate equation. 

Equation (5a) gives the formula for estimating the total cost of employing the 
present selection procedure to meet the quota of satisfactory personnel : 

TC p = [ (NRXCR) ] + [ (NR)(CP) 1 + [ (NA)(CI) j 

+ [ (NA)(CT) ] + [ (NEA)(CEA) + (NER)(CER) ] (5a) 

where TC p is the total cost of using the present selection procedure to meet the quota, 
CR is the cost of recruiting a person, 

CP is the cost of administering the present procedure, 

Cl is the cost of inducting a person, 

CT is the cost of training a person, 

CEA is the cost of an erroneous acceptance, 

CER is the cost of an erroneous rejection, and the others are the same as above. 
Substituting the data of the present ROKAF selection procedure into Equation 
(5a) gives: 



TC p = [ (100)(S4000) ] + [ ( 100)(S 1 500) ] + [ (100)(S100) ] 
+ [ (100)(S35000) 1 + [ (50)(S8000) + (0)(S1500) 1 
= 53,619,120 



Equation (5b) gives the formula for estimating the total cost of employing the 
experimental selection procedure to meet the quota : 

TC e = [ (NR)(CR) ] + ( (NRXCP) ] + [ (NR)(CE) ] + [ (NA)(CI) ] 

+ [ (NA)(CT) ] + [ (NEAXCEA) + (NER)(CER) ] (5b) 

where TC g is the total cost of using the experimental selection procedure to meet the 
quota, CE is the cost of administering the experimental procedure, and the others are 
the same as above. 

Using Equation (5b) for the experimental selection procedure ( /= 8) gives: 
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TC e = [ (109)(S4000) ] + [ ( 109)(S 1 500) ] + [ (109)(S80) ] + [ (79)(S100) ] 
+ [ (79)(S35000) ] + [ (29XS8000) + (4XS1500) | 

= 5751,920 



A useful program for various data values is provided in Appendix B and creates a 
sample CAPER model output data for ROKAF in Table 9 for the different values. 
This output contains five types of information : 

1) number recruited, 

2) number of erroneous acceptance, 

3) number of erroneous rejections, 

4) number accepted, 

5) total cost. 

These five estimates may suffice for many personnel program managers. And also 
contains cost for each of the following: 

1) recruiting 

2) selection 

3) induction 

4) training, and 

5) erroneous decisions. 

Table 9 shows that as the cutting score increases the subsequent results are as 
follows : 

1) a greater number of persons must be recruited, 

2) a smaller number of persons is accepted, 

3) the number of erroneous acceptances decreases, and 

4) there is an increase in erroneous rejections. 

These four consequences have cost complications. As recruiting and selection costs 
increase, induction and training costs are decreased. The cost of erroneous acceptances 
and erroneous rejections, decreases at first, hits the cutting score that minimizes the 
sum of both costs ( /= 11) and then increases as the cutting score is raised farther and 
the total cost changes similarly. Figure 4.4 graphically shows this changes of total cost 
as increasing the cutoff scores. 

If the ROKAF wants to achieve 50 graduates at a minimum total cost, 
comparing the estimated total cost of the present selection procedure (S4, 460, 000) in 
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TABLE 9 

EXAMPLE OF CAPER MODEL OUTPUT DATA OF ROKAF 



1 

1 

1 

1 

+ 


CUT 

SCORE 


1 NO l 
1 REC. | 


NO | 
ACC. 1 


NO 

ERR. 

ACC. 


T 

l 

1 

l 


NO | 
ERR. | 
1 

REJ . | 






COST 










1 

1 


RECRUIT 


SELECT | 


INDUCT 


I 


TRAINING I 


ERR. DEC. 


1 


TOTAL 


“ 1 
1 

+ 


1 


NA 


1 100 | 


100 1 


50 


1 

i 


0 1 
1 


900000 . 


150000. | 


1 0000 


• 1 


3500000 . 


900000 . 


1 

i 


9960000 . 


1 

i 


1 


0 


100 l 


100 1 


50 


1 


0 1 


900000 . 


153000. | 


10000 


. 1 


3500000 . 


900000. 


1 

1 


9963000 . 


1 

1 


1 


1 


1 100 I 


100 I 


50 


l 


0 i 


900000 . 


153000. | 


1 0000 


. 1 


3500000 . 


900000 . 


1 


9963000 . 


1 


1 


2 


1 100| 


99 1 


99 


l 


0 1 


900000 . 


153000. | 


9900 


• 1 


3965000. 


392000 . 


1 


9929900 . 


1 


1 


3 


I 100 | 


93 1 


93 


l 


o 1 


900000 . 


153000. | 


9300 


• 1 


3930000 . 


339000 . 


1 


9331300 . 


1 




A 


I 100 | 


97 | 


97 


1 


0 1 


900000 . 


153000. I 


9700 


. 1 


3395000. 


376 0 00 . 


1 


9333700 . 


1 




5 


1 100 | 


99 | 


99 


1 


0 1 


900000 . 


153000. | 


9900 




3290000. 


352000. 




9209900 . 


1 




6 


1 102 1 


90 | 


90 


1 


1 1 


9C3000 . 


161160. | 


9000 




3150C00 . 


321500 . 


1 


9099660 . 


1 


1 


7 


1 109 | 


35 I 


35 


1 


2 1 


9160C0. 


169320. | 


8500 




2975000. 


283000 . 


1 


3396320 . 


1 


1 


3 


1 109 | 


79 I 


29 


l 


9 I 


936000 . 


172220. I 


7900 


! i 


2765000. 


238000 . 


1 


3619120 . 


1 


1 


9 


1 113 | 


73 1 


23 


l 


9 1 


972000 . 


• 136990. | 


7 300 




2555000 . 


197500 . 


1 


3913290 . 


1 


1 


10 


1 139 | 


67 | 


17 


1 


17 | 


536000 . 


211720. 1 


67 00 




2395000 . 


16 1500 . 


1 


3260920. 


1 


1 


11 


1 163 1 


62 1 


12 




32 I 


652000 . 


257590. | 


6200 


! i 


2170000 . 


199000 . 


1 


3229790. 


1 


1 


12 


1 219 | 


59 I 


9 


1 


57 I 


356000 . 


333120. | 


5900 




206 5000 . 


157500 . 


1 


392252C. 


1 


1 


13 


1 301 1 


56 1 


6 


l 


101 I 


1209000 . 


975530. | 


5600 




1960000 . 


199500. 


1 


3399630 . 


1 




19 


1 963 | 


59 | 


9 


1 


131 | 


1S52000 . 


731590. | 


5900 




1390000 . 


303500. 


1 


9732990 . 


1 


1 


15 


1 799 | 


52 1 


2 


l 


397 I 


3176000 . 


1259520. 1 


5200 




1320000 . 


536500 . 


1 


6792220. 


1 


1 


16 


1 1971 1 


50 1 


0 


1 


635 | 


5839000 . 


2329180. 1 


5000 




1750000 . 


1027500. 


1 


10990630. 


1 


1 


17 


1 2991 | 


50 I 


0 




1921 1 


11769000 . 


9696730 . | 


5000 




1750000 . 


2131500 . 


1 


20297230 . 


1 


1 

+ 


13 


1 7193 | 


50 I 


0 


1 


3521 1 


23572000. 


11285990. | 


5000 




1750000 . 


5281500 . 


1 


96399916 . 


1 

+ 



Table 9 suggest that using the experimental testing procedure with a cutting score of 8 
(S3,619,120) would be cost effective which is the same result as the utility evaluation. 

Table 9 shows that the minimum total cost of attaining the quota, 50 graduates, 
is S3, 229, 740 using a cutting score 11 on the test and the optimal recruiting-selection 
strategy is to recruit 163 persons. The best estimate is that 62 of these persons will 
qualify for acceptance and 12 of the selectees can be expected to fail the training 
course, leaving the 50 graduates required to meet the required quota. In comparison 
with the present selection procedure, the use of the experimental selection procedure 
will save an estimated SI, 230, 260 or S24,605.20 per graduate. 
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Figure 4.4 Graphical view of CAPER model output. 
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V. DISCUSSION 



The increasing number of pilot candidates entering the ROKAF has resulted in 
increasing costs due to high attrition rates and decreasing quality of pilots who do 
complete the program. Further, training funds and personnel resources remain limited. 
Solving these problems will not be an easy task. It appears that there is not a single 
simple solution. However, it is imperative that some restructuring of the selection 
process be initiated as soon as possible as the potential negative impact these problems 
will have continues to increase. It is hoped that the current "Operations with 
Economic Efficiency (OEE) Program", which is attempting to maximize the cost 
effectiveness of all ROKAF operations will be able to adequately address the pilot 
selection problems. This thesis represents an attempt to suggest an effective method 
for improving the selection process. 

As was discussed in Chapter 1, other nations - France, Italy and England - 
initiated the selection testing efforts as early as World War I and the USN (USAF) 
reduced the attrition rate from 60% to 30% (25%) using psychological testing in the 
early 1970s. The Pensacola 1000 aviator study found that psychological and 
psychomotor measures had more validity for the prediction of success in flight training 
than did physiological measures. The psychomotor testing scores are significantly 
related to flight training pass/fail criteria. The ROKAF pilot selection system, 
however, only depends on actual flight performance. 

As we evaluated its potential benefits to the ROKAF in Chapter IV, the expected 
utilities and CAPER model, could have a significant impact on pilot selection, pilot 
training and aircraft accidents. In terms of the analytic CAPER model, where output 
data are fixed by the values of the input data, the utility of the CAPER output can be 
no better than the accuracy of the input data. If the cost estimates are inaccurate, cost 
forecasts and the optimal recruiting-selection strategy will be distorted. This may not 
be a serious problem since the approximations are usually adequate for the kinds of 
decisions that estimates are used to make, and there is no need for utility estimate 
down to the last dollar. 

It has been shown that a greater number of pilot candidates attrite during actual 
flight training than during the pre-training medical screening process. These 
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candidates, though often having the option to continue their military careers in other 
aviation related fields, frequently feel that they are inferior officers because of the 
failure during flight training. These feelings of low self esteem, morale and motivation 
often result in poor performance and the involuntary termination of the officer's career. 
It is suggested that if a viable psychological test battery was administered in addition 
to medical screening and a candidate failed that test then (1) there would be a 
significant savings in actual flight training time for that candidate and (2) his being 
identified as "not aeronautically adaptable" might not have the psychological impact as 
having failed because of poor performance during actual training flights. 

A. RECOMMENDATIONS 

This study which is one of the OEE programs planned to reduce the cost and 
improve the efficiency of all ROKAF operations was essentially designed to emphasize 
the importance and necessity of mental testing methods on pilot selection for 
successfully screening out potential failures and suggests useful pilot screening methods 
to ROKAF. Also provided are evaluation techniques for decision makers which are 
the decision strategies and CAPER model. 

It is clear from the overall study results that the use of psychological tests or test 
batteries of other nations can improve the prediction outcomes of the ROKAF pilot 
candidate selection method. Many other national Air Forces have used psychomotor 
screening to select their pilot candidates and continue efforts to develop this 
methodology. Some of the psychomotor scores are significantly different between 
candidates who graduate from UPT and those who are eliminated. The ideal selection 
system, however, would measure and consider all relevant characteristics of a 
candidate. The UPT selection system currently used by the ROKAF does not consider 
all relevant information about candidates and many candidates fail training because of 
deficiencies in flying aptitude. The intense situation of ROKAF, moreover, requires 
selecting high quality student pilots and the candidates with the lowest probability of 
success would be screened out before training. The higher the cutoff point is set, the 
better the pilot quality, cost effectiveness not with standing. The "false rejection", 
however, is a problem with all screening systems, including flight screening, and must 
be evaluated in terms of the percentage screened and the quality of the graduating 
students. 

As suggested in Chapter IV, a change using experimental testing on ROKAF 
pilot selection method is feasible. It suggests a savings of S24,605.20 per graduate in 
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our example. Multitask performance is useful .for the primary flight training 
performance prediction. The experiment takes only 20 minutes and is easy to 
administer. The psychomotor screening system used by the USAF, which includes 
Two-Hand Coordination and Complex Coordination, has been shown to be useful in 
screening not only candidates for UPT but also for identifying those who demonstrate 
a potential for successfully completing advanced fighter training. Any of three 
Integrated Pilot Candidate Selection Models could be used to select UPT candidates in 
the same manner as the psychomotor screening system. The whole person concept is 
imbedded in the multidimensional IPCSM approach to pilot selection. 

The implementation of psychomotor screening and multitask performance will 
improve the quality of student candidates and reduce attritions in ROKAF UPT. 
Based on overall considerations, however, an integrated selection system including 
psychomotor screening is recommended for ROKAF. 

One of the preferable pre-selection systems that suggests for ROKAF pilot 
selection system of AFA cadets in this study would be: 

1) The General Staff defines the number of pilots needed, and a circular is issued 
identifying the number of candidates required. 

2) The selection process includes: 

a) Preliminary Examination - Medical 

b) EEG recordings (Electroencephalogram) 

c) Integrated Psychological Test 

d) Individual interview. 

3) Cadets must wait until after their graduation from the KAFA and commission 
as officers before starting flight training. 

Other useful methods which might also be incorporated in a pre-selection screening 
process would be: 

• Perceptual/Cognitive Paper-and-Pencil Tests, 

• Neurological Test, 

• Residual Attention Test. 

The Residual Attention Test could be especially valuable in predicting how well an 
individual will handle emergency situations. This could potentially have an impact on 
the aircraft accident rate in accidents due to human error, incorrect procedures or just 
poor handling of the aircraft during emergencies. It is obvious that with the many 
types of individual psychological and psychomotor tests available that further research 
must now address which individual tests and combinations of tests and batteries of 
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must now address which individual tests and combinations of tests and batteries of 
tests should be incorporated into the ROKAF selection program. It is strongly 
recommended that a ROKAF pilot selection program research and development (R & 
D) team be organized to address this and other selection issues. The team could also 
benefit greatly from a close liaison with U.S. Navy’ and Air Force R & D centers 
currently engaged in aircrew selection methodology. 

It is hoped that this study will be useful in stimulating the ROKAF in developing 
improved selection methods and in predicting success in training as a means of 
increasing aircrew training program cost effectiveness. 
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List of Abbreviation 



ACTB 


Aircrew Classification Test Battery 


AFA 


Air Force Academy 


AFB 


Air Force Base 


AFHRL 


Air Force Human Resources Laboratory (AFSC) 


AFOQT 


Air Force Officer Qualifying Test 


AFROTC 


Air Force Reserve Officer Training Corps 


AOC 


Aviator Officer Candidate 


AQT 


Aviation Qualification Test 


ASTB 


Aviator Selection Test Battery 


FAR 


Flight Aptitude Rating 


FR 


Fighter Recommended 


FSP 


Flight Screening Program (OTS) 


IPCSM 


Integrated Pilot Candidate Selection Model 


KAFA 


Korea Air Force Academy 


OEE 


Operations with Economic Efficiency 


OTS 


Officer Training School 


ROKAF 


Republic of Korea Air Force 


R&D 


Research and Development 


UPT 


Undergraduate Pilot Training 


L'SAF 


United States Air Force 


USN 


United States Navy 
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APPENDIX A 

ATTRITION CATEGORIES 



A student pilot can be dropped from a training program for several reasons. It is 
agreed that there is no one simple causative factor nor a quick and easy solution. 
Some of the primary reasons for student attrition are : 

1. Voluntary Withdrawal or Drop On Request (DOR) 

Over 50 research reports and findings associate the main reason of DOR 
with anxiety, tension or fear. Most of the specific reasons for the DOR are expressed 
an anxiety or tension with regard to flying, fear of flying, or lack of confidence in 
handling aircraft. Also mentioned are a dislike or distaste of flying and a shift to a 
civilian career interest. Some of the researches indicate a connection of fear of aircraft 
accidents or crashing with failure in training. 

2. Not Phisically Qualified 

Most research suggests that "a person with anxiety may see the flight 
surgeon when he has a symptom which he knows might result in his being dropped 
from the program; whereas, a nonanxious, highly motivated person might never go to 
the flight surgeon." 

3. Flight Failure (FF) 

Figure 1.1 mentions the types of attrition by stage of training. The data 
provide dramatic evidence of the relationship of DOR and FF in primary training and 
the transition phase. 

The research literature generally supports the view that anxiety is both a 
cause and a symptom of inadequate or inferior performance in training. It can be 
broken down into two subcategories: 

1) A student group prone to anxiety or with anxiety predisposition. 

2) A student group going through a transient anxiety producing situation. 

4. Not Officer Material 

Is a statistic more oriented to the time during which attrition occurs rather 
than a specific type of attrition. 

5. Academic Failure 

Unsatisfactory performance in academic work. 
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6. Transfer to Another Training Program 

This is a small percentage of the total attrition and means a request to 
change one's specialty. 

7. Practical Work Failure 

Unsatisfactory performance in a ground simulator or training device. 
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APPENDIX B 

CAPER MODEL PROGRAM 



************************************************************************ 



* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 



CAPER MODEL 

This program provides a CAPER MODEL OUTPUT as various input data. 
The program reads data from data file, CAPER DATA, and creates a 
formated output, CAPEROU LISTING on your disk automatically using 
exec file. The data used are from tne Table 8 and there is no 
no meaning of ROKAF data but only for showing how the program works. 

The key variables used are : 



* 

* 

* 

* 

* 

★ 

★ 

★ 

★ 

★ 

★ 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

************************************************************************ 

PARAMETER (N=20) 

REAL PG(N) f PF(N) 

DATA NQ/ 50/ ,BR/ . 5/ . CR/4000 . / , CP/ 1500 . / , CI/100 . / ,CE/80 ./ ,CT/35000 . / 
* CEA/8000/ ^ER/ISOO/ 

*** READ DATA FROM DATA FILE *** 



NR 

NA 

NEA 

NER 

NQ 

BR 

PG 

PF 

CR 

CP 

CE 

Cl 

CT 

CEA 

CER 

TCP 

TCE 



number recruited 
number accepted 

number of erroneous acceptances 
number of erroneous rejections 
the quota of satisfactory personnel 
base rate 

proportion of graduates who would qualify for 

acceptance at the ith cutting score on test 

proportion of failures who would qualify for 

acceptance at the ith cutting score on test 

cost of recruiting a person 

cost of administering the present procedure 

cost of administering the experimental procedure 

cost of inducting a person 

cost of training a person 

cost of an erroneous acceptance 

cost of an erroneous rejection 

total cost of using the present selection 

procedure to meet the quota 

total cost of using the experimental selection 
procedure to meet the quota 



This program could be used effectively for other CAPER MODEL 
changing only the data and also useful for interactive program with 
some developments. 
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DO 10 1=1. N 

READ (1,5) PG(I),PF(I) 
CONTINUE 



*** OUTPUT FORMAT *** 



65 

75 



WRITE l 


3 ' 15 


WRITE l 


3,65) 


FORMAT ( 


, 1X L ( 


WRITE i 


[3 ,*75 ) 


FORMAT ( 


.IX, ’ | 


*) 

WRITE* 


(3,*) 


WRITE* 


3,25) 


WRITE* 


(3 , *) 


*ECT | 


IN 



CUT | NO | NO 
SCORE | REC. | ACC. 

i i 



■ i 
1 1 



| NO 
| ERR. 

I 



| TRAINING | 



NO | 1 , 72X, ' | ' ) 

ERR. | ' ,34X, 'COST' ,34X, ' | ' 

I r 

RECRUIT | SEL 



| ACC. | REJ. | 

ERR. DEC. | TOTAL | 
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WRITE (3 ,15) 

WRITE (3,*) 

*** CALCULATE TOTAL COST *** 



DO 20 J=1,N 

NR = NINT(NQ/(BR * PG(J))) 
NEA = NINT (NR * (1 
NER = NINT (NR * BR 



NA 
TR 
SP 
SE 
TI 
TT 
TE 
TCS = 



NQ 

NR 

NR 

NR 

NA 

NA 

NEA 

TR 



NEA 

CR 

CP 

CP + 

Cl 

CT 

* CEA 
+ TI H 



- BR) * PF ( J) ) 
' (1 - PG(J)>> 



NR 



CE 



+ NER 
TT + 



* CER 
TE 



TCP = TCS + SP 
TCE = TCS + SE 
IF (J EQ 1) THEN 

Write (3, 35) nr,na,nea,ner,tr,sp,ti,tt,te,tcp 

WRITE(3, 55) 

ELSE 

WRITE (3, 45) (J-2) ,NR,NA,NEA,NER,TR,SE,TI ,TT,TE,TCE 
END IF 



20 


CONTINUE 




WRITE (3 , 15) 


5 


FORMAT (F5.3 


15 


FORMAT (IX, 1 


25 


FORMAT ( 1 + 1 , 


35 


FORMAT (IX, 1 


45 


FORMAT (IX, 1 


55 


FORMAT (IX, 1 



, 5X, F5 . 3) 



STOP 

END 



T37 , 72 ( 
NA | 

1 , 12 , 
,T8 , 1 



,4 14, , 

I ',4(14, 
1 ,4(4X, 1 I 



').6(F10.0,' I 1 )) 
1 1 ,6 F10.0 ' 

■ > ,6(10X, ■ |'J) 



)) 



*** DATA FILE *** 
PG PF 



1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


.986 


1.000 


.966 


1.000 


.932 


.996 


.874 


.984 


.784 


.962 


.668 


.920 


.532 


.848 


.388 


.744 


.256 


.612 


.152 


.468 


.080 


.332 


.038 


.216 


.016 


.126 


.004 


.068 


.000 


.034 


.000 


.014 


.000 
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APPENDIX C 

TAYLOR AND RUSSELL TABLES 

N. M. Absahaus, E. F. Alf, and J. H. Wolf* 



TABLE 1 

Peotoetion Who Well Bz Satistactoiy Amdno Thou Selected, roi Grvzw values or 
thz PaopomoN or Pixscrr Employees Considzixd Satisjactoey, rax 
Szixctiom Ratio, a*® Poorr-BnxEiAL r 



Selection ratio 

























.050 


.100 


.200 


.500 


.400 


J00 


.600 


.700 


.800 | .900 



Proportion of employee* considered satisfactory - .05 



.00 


.050 


.050 


.050 


.050 


.050 


.050 


.050 


.050 


.050 


.050 


.050 


.05 


.076 


.072 


.066 


.063 


.061 


.059 


.057 


.053 


.054 


.052 


.051 


.10 


.112 


.099 


.086 


.078 


.072 


.067 


.063 


.060 


.056 


.053 


.052 


.15 


.157 


.131 


.106 


.092 


.082 


.075 


.068 


.063 


.058 


.054 


.052 


.20 


.215 


.169 


.128 


.107 


.092 


.082 


.073 


.066 


.060 


.055 


.052 


25 


.278 


.210 


.151 


.121 


.101 


.087 


.076 


.068 


.061 


.055 


.052 


.30 


J52 


.255 


.173 


.133 


.109 


.092 


.079 


.070 


.062 


.055 


.053 


.55 


.433 


.502 


.194 


.144 


.113 


.095 


.081 


.070 


.062 


.056 


.053 


.40 


JI9 


.348 


.212 


.153 


.119 


.097 


.082 


.071 


.062 


.056 


.055 


.45 


.607 


.393 


.227 


.159 


.122 


.099 


.083 


.071 


.062 


.056 


.053 


.50 


.6*4 


.431 


.238 


.163 


.124 


.100 


.083 


.071 


.062 


.056 


.053 


.55 


.776 


.462 


.245 


.165 


.124 


.100 


.083 


.071 


.062 


.056 


.053 


.60 


.849 


.484 


.248 


.166 


.125 


.100 


.083 


.071 


.062 


.056 


.053 


.65 


.909 


.495 


.250 


.167 


.125 


.100 


.083 


.071 


.062 


.056 


.053 


.70 


.954 


.499 


.250 


.167 


.125 


.100 


.083 


.071 


.062 


.056 


.053 


.75 


.982 


.500 


.250 


.167 


.125 


.100 


.083 


.071 


.062 


.056 


.053 


.80 


.996 


.500 


.250 


.167 


.125 


.100 


.083 


.071 


.062 


.056 ' 


053 


.85 | 


.996 


.500 


.250 1 


.167 


.125 


.100 


.083 


.071 


.062 


.056 


.053 


.90 


.996 


.500 


.250 


.167 


.125 


.100 


.083 


.071 


.062 


.056 


.053 


.95 


1.000 


.500 


-250 


.167 


.125 


.100 


.083 


.071 


.062 


.056 


.053 


1.00 


1.000 

1 


.500 


.250 


.167 


.125 


.100 


.083 




.062 


.056 


.05- : 



Proportion of employees considered satisfactory » .10 



.00 


.100 


.100 


.100 


.100 


.100 


.100 


.100 


.100 


.100 


.100 


.100 


.05 


.134 


.129 


.122 


.118 


.115 


.112 


.110 


.107 


.105 


.103 


.102 


.10 


.176 


.162 


.147 


.137 


.130 


.124 


.118 


.114 


.109 


.105 


.103 


.15 


.226 


200 


.173 


.157 


.145 


.135 


.127 


.120 


.113 


.107 


.104 


20 


.284 


.243 


202 


.177 


.160 


.146 


.135 


.125 


.116 


.108 


.104 


25 


J49 


-290 


232 


.198 


.175 


.157 


.142 


.130 


.119 


.109 


.104 


JO 


.422 


J43 


.264 


.219 


.189 


.166 


.148 


.133 


.121 


.110 


.105 


JS 


J02 


J99 


.297 


.240 


.202 


.175 


.153 


.136 


.122 


.110 


.105 


.40 


J88 


.460 


J30 


.260 


-214 


.182 


.158 


.139 


.123 


.111 


.105 


.45 


.676 


.525 


J64 


.278 


225 


.188 


.161 


.140 


.124 


.111 


.105 


JO 


.765 


.592 


J97 


295 


234 


.193 


.164 


.142 


.125 


.111 


.105 


J3 


M7 


.661 


.427 


J09 


240 


.196 


.165 


.142 


.125 


.111 


.105 


.60 


.917 


.731 


.454 


J20 


245 


.198 


.166 


.143 


.125 


.111 


.105 


.65 


.966 


.800 


.476 


J27 


24A 


.199 


.166 


.143 


.125 


.111 


.105 


.70 


.991 


.864 


.490 


J31 


.249 


.200 


.167 


.143 


.125 


.111 


.105 


.75 


.999 


.920 


.497 


J33 


.250 


200 


.167 


.143 


.125 


.111 


.105 


.80 


1.000 


.964 


.500 


J33 


.250 


200 


.167 


.143 


.125 


.111 


.105 


.85 


1.000 


.990 


J00 


.333 


-250 


200 


.167 


.143 


.125 


.111 


.105 


.90 


1.000 


.999 


J00 


J33 


25 0 


200 


.167 


.143 


.125 


.111 


.105 


.95 


1.000 


.999 


.500 


.333 


-250 


200 


.167 


.143 


.125 


.111 


.105 


1.00 


1.000 


1.000 


.500 


J 33 


.250 


200 

t 


.167 


.143 


.125 


.111 


.105 
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Taylor- Russell Tables for Dichotomous -Criterion Variables 



TABLE 1 — {Continued) 



Selection ratio 



r p6 


.050 


.100 


.200 


J00 


.400 


.500 


.600 


.700 


.800 


.900 


.950 


Proportion of employees considered satisfactory* - .20 


.00 


.200 


.200 


.200 


.200 


.200 


.200 


.200 


.200 


.200 


.200 


.200 


.05 


.244 


.237 


.229 


.224 


.220 


.216 


.213 


.210 


.207 


.204 


.202 


.10 


.293 


.277 


.260 


.248 


.239 


.232 


.225 


.219 


.213 


.207 


.204 


.15 


.347 


( .320 


.292 


.274 


.260 


.248 


.238 


.228 


.219 


.210 


J205 


.20 


.405 


.368 


.326 


.300 


.280 


.264 


.249 


.236 


.224 


.213 


.207 


.25 


.468 


.418 


.363 


.328 


.301 


.280 


.261 


.244 


.229 


.215 


.208 


.30 


.535 


.472 


.401 


.356 


.322 


.295 


.272 


.252 


.234 


.217 


.208 


.35 


.605 


.529 


.441 


.385 


.344 


.311 


.283 


.259 


.238 


.218 


.209 


.40 


.676 


.589 


.484 


.416 


.366 


.326 


.293 


.265 


.241 


.220 


.210 


.45 


.747 


.652 


.529 


.448 


.387 


.340 


.302 


.270 


.244 


.220 


.210 


.50 j 


.815 


.717 


.577 


.480 


.409 


.354 


.310 


.275 


.246 


.221 


. -.210 


.55 ‘ 


.876 


.782 ; 


.627 


.514 


.430 | 


.367 


.318 


' .279 


.248 


.222 


.210 


.60 


.927 : 


.846 


.680 


.547 1 


.450 1 


.378 


.324 


.282 


.249 


.222 


.210 


.65 


.964 , 


.904 , 


.736 


.581 ’ 


.468 


.387 


.328 


.284 


.249 


.222 


.210 


.70 


.986 


.952 , 


.794 ( 


.612 ; 


.482 


.394 


.331 


.285 


.250 


.222 


.210 


.75 


.996 


.983 


.852 


.639 


.492 


.398 


.333 


.286 


.250 


.222 


.210 


80 


1.000 


.997 


.909 


.657 i 


.498 


.400 


.333 


.286 


.250 


.222 


.210 


85 


1.000 


L000 


.958 


.665 J 


.500 


.400 


.333 


.286 


.250 


.222 


.210 


90 


1.000 


1.000 1 


.990 


.667 j 


.500 


.400 1 


.333 


.286 , 


.250 


.222 | 


.210 


.95 


1.000 


1.000 


.990 


.667 j 


.500 1 


.400 


.333 , 


.286 


.250 


.222 


.210 


1.00 

1 


1.000 


1.000 


1.000 


667 1 

I 


.500 

1 


.400 | 


.333 


.286 


.250 


.222 ! 


.211 



Proportion of employees considered satisfactory * .30 



.00 


.300 


.300 


.300 


1 .300 


1 

; .300 


.300 


> .300 


.300 


.300 


.300 


.300 


.05 


.349 


.341 


.333 


i .327 


! .322 


.318 


.315 


.311 


! .308 


.304 


1 .302 


.10 


.401 


.385 


367 


.354 


.345 


.337 


.329 


.322 


.316 


.308 


.305 


.15 


' .456 


.430 


j .402 


! .383 


.368 


! .355 


.344 


.333 


| .323 


.312 


.307 


.20 


.513 


.478 


.438 


.412 


.391 


.374 


.358 


.344 


.330 


.316 


.308 


.25 


.572 


.528 


.476 


.442 


.415 


.393 


.372 


.354 


1 .336 


.319 


.310 


30 


1 .632 


.579 


.516 


.473 


.440 


.412 


.387 


.364 


.343 


.322 


.311 


35 


.692 


.632 


.557 


.506 


.465 


.431 


.401 


.374 


.349 


.324 


.312 


40 


.751 


.686 


.600 


.540 


.492 


.451 


.415 


.383 


.354 


.327 


.314 


.45 


.808 


1 .740 


.646 


.576 


.519 


.471 


.429 


.392 


.359 


.329 


.314 


.50 


.860 1 


1 .794 


.693 


.614 


.547 


.491 


.443 


.401 


.363 


.330 


MS 


.55 


.905 


.846 


.742 


.653 


.577 


.512 


.456 


.408 


.367 


.331 | 


.315 


.60 


.942 


.894 


.794 


.696 


.608 


.532 


.468 


.415 


.370 


.332 


.316 


.65 


.970 1 


.936 


.846 


.741 


.639 


.552 


.479 


.420 


.372 


.333 


.316 


.70 


.987 1 


.967 


.897 


.789 | 


i .671 


.570 


.488 


.424 


.374 


.333 


.316 


.75 


.996 


.988 


.944 


.839 | 


.702 


.584 


.495 


.427 


.375 


.333 


.316 


.80 


.999 


.997 


.979 


.891 


.728 


.594 


.498 


.428 


.375 


.333 


.316 


.85 


1.000 


1.000 


.997 


.941 1 


.745 


.599 


.500 


.428 


.375 


.333 


.316 


.90 


1.000 


1.000 


1.000 


.982 1 


.750 


.600 


.500 i 


.429 


.375 


.333 I 


.316 


.95 


1.000 


1.000 


1.000 j 


.982 


.750 


.600 | 


.500 


.429 


.375 


.333 


.316 


1.00 


1.000 


1.000 


1.000 


1.000 | 


.750 


.600 


.500 


.429 


.375 


.333 1 


.316 



Proportion of employees considered satisfactory * .40 



.00 


.400 


.400 


.400 


.400 


.400 1 


.400 


.400 


.400 


.400 


.400 1 


.400 


.05 


.451 


.444 


.435 


.429 


.424 


.420 


.416 


.412 


.408 


.405 1 


.403 


.10 


.504 


.488 


.470 


.458 


.448 


.439 


.432 


.424 


.417 


.409 ! 


.405 


.15 


.557 


.534 


.506 


.487 


.472 


.459 


.447 


.436 


.425 


.414 


.408 


.20 


.611 


.580 


.542 


.517 


.497 


.479 


.463 


.448 


.433 


.418 | 


.410 
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N. M. Abrahams, E. F. Axf, and J. H. Wolfe 

TABLE 1 — (Continued) 



Selection ratio 





.050 


.100 


.200 


.300 


.400 


.500 


.600 


.700 


.800 


.900 

1 


.950 








Proportion of employees considered 


satisfactory - .40 






.25 


.665 


.626 


.580 


.548 


.522 


.500 


.479 


.460 


.441 


.422 


.412 


.30 


.717 


.673 


.618 


.580 


.548 


.521 


.496 


.472 


.449 


.426 


.414 


.35 


.768 


.720 


.658 


.613 


.575 


.542 


.512 


.484 


.456 


.429 


.415 


.40 


.816 


.766 


.698 


.647 


.603 


.564 


.529 


.496 


.464 


.433 


.417 


.45 


.860 


.811 


.740 


.682 


.632 


.588 


.546 


.508 


.471 


.436 


.418 


.50 


.900 


.854 


.782 


.720 


.663 


.612 


.564 


.519 


.477 


.438 


.419 


.55 


.933 


.894 


.824 


.758 


.696 


.637 


.582 


.530 


.483 


.440 


.420 


.60 


.959 


.929 


.866 


.799 


.730 


.663 


.600 


.541 


.489 


.442 


.420 


.65 


.978 


.958 


.906 


.842 


.768 


.691 


.617 


.551 


.493 


.443 


.421 


.70 


.990 


.979 


.943 


.885 


.807 


.719 


.634 


.560 


.496 


.444 


.421 


.75 


.997 


.992 


.972 


.928 


.850 


.748 


.649 


.566 


.498 


.444 


.4 2i 


.80 


.999 


.998 


.991 


.966 


.894 


.774 


.660 


.570 


.500 


.444 


.42. 


.85 


1.000 


1.000 


.998 


.992 


.939 


.792 


.665 


.571 


.500 


.444 


.421 


.90 


1.000 


1.000 


1.000 


1.000 


.979 


.800 


.667 


.571 


.500 


.444 


.421 


.95 


1.000 


1.000 


1.000 


1.000 


.979 


.800 


.667 


.571 


.500 


.444 


.421 


1.00 


1.000 


1.000 


1.000 


1.000 


1.000 


.800 


.667 


.571 


.500 


.444 


.421 



Proportion of employees considered satisfactory * .50 



.00 


.500 


.500 


.500 


.500 ' 


.500 


1 

.500 


.500 


.500 


.500 


.500 


1 .500. 


.05 


.552 


.544 


.535 


.529 ; 


.524 


.520 


.516 


j .512 


.509 


.505 


i .503 


.10 


.602 


.588 


.570 


.558 1 


.548 


.540 


.532 


.525 


.518 


.510 


.505 


.15 


.653 


.631 


.605 


.587 1 


1 .573 


.560 


.549 


1 .538 


.526 


.515 


1 .50$ 


.20 


.702 


.674 


.640 


.617 I 


.598 


.581 


.565 


.550 


.535 


.519 


.5:1 


.25 


.748 


.716 


.676 


.647 


.623 


.602 


.582 


.563 


.544 


.524 


.513 


.30 


.793 


.757 


.712 


.678 


.649 


.623 


.599 


.576 


.553 


.529 


.515 


.35 


.834 


.797 


.747 


.709 


.676 


.646 


.617 


.590 


.562 


.533 


.518 


.40 


.872 


.835 


.783 


.741 


.703 


.669 


.636 


.603 


.571 


.537 


.520 


.45 


.905 


.871 


.818 


.773 


.732 


.693 


.655 


.617 


.580 


.541 


.521 


.50 


.933 


.903. 


.853 


.807 


.762 


.718 


.675 


.631 


.588 


.545 


.523 


.55 


.956 


.932 


.886 


.840 


.793 


.745 


.696 


.646 


.597 


.548 


.524 


.60 


.974 


.956 


.918 


.875 


.826 


.773 


.718 


.660 


.604 


.551 


.525 


.65 


.986 


.975 


.946 


.908 


.861 


.804 


.740 


.675 


.612 


.553 


.526 


.70 


.994 


.988 


.969 


.940 


.896 


.836 


.764 


.689 


.617 


.554 


.526 


.75 


.998 


.995 


986 


.968 


.932 


.872 


.788 


.700 


.622 


.555 


.526 


.80 


1.000 


.999 


.996 


.988 


.966 


.909 


.810 


.709 


.624 


.555 


.526 


.85 


1.000 


1.000 


.999 


.998 


.990 


.947 


.827 


.713 


.625 


.556 


.5-6 


.90 


1.000 


1.000 


1.000 


1.000 


.999 


.980 


.833 


.714 


.625 


.556 


.5-6 


.95 


1.000 


1.000 


1.000 


1.000 


1.000 


.999 


.833 


.714 


.625 


.556 


.526 


1.00 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


.833 


.714 


.625 


.556 


.526 



Proportion of employees considered satisfactory - .60 



.00 


.600 


.600 


.600 


.600 


.600 


.600 


.600 


.600 


.600 


.600 


.600 


.05 


.650 


.642 


.634 


.628 


.624 


.620 


.616 


.612 


.609 


.605 


.603 


JO 


.697 


.684 


.667 


.656 


.647 


.639 


.632 


.625 


.618 


.610 


.606 


J5 


.742 


.723 


.700 


.684 


.671 


.659 


.648 


.637 


.626 


.615 


.608 


JO 


.784 


.761 


.733 


.712 


.695 


.679 


.664 


.650 


.636 


.620 


.611 


.25 


.824 


.798 


.764 


.740 


.719 


.700 


.681 


.663 


.645 


.625 


.614 


JO 


.859 


.832 


.796 


.768 


.743 


.721 


.699 


.677 


.655 


.630 


.617 


.35 


.891 


.864 


.826 


.796 


.768 


.742 


.717 


.691 


.664 


.636 


.619 


.40 


.918 


.893 


.855 


.823 


.793 


.764 


.735 


.706 


.675 


.641 


.622 


.45 


.942 


.920 


.883 


.851 


.819 


.788 




.721 


.685 


.646 


.624 
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Taylor-Russell Tables for Dichotomous. Criterion Variables 



TABLE 1 — (Continued) 



Selection ratio 



fpb 





.050 


.100 


.200 


J00 


.400 


.500 


.600 


.700 


J .800 


.900 


.950 


Proportion of employees considered satisfactory * .60 


.50 


.961 


.942 


.910 


.878 


.846 


.812 


.776 


.737 


.695 


.650 


.626 


.55 


.976 


.961 


.934 


.905 


.872 


.837 


.797 


.754 


.706 


.655 


.628 


.60 


.986 


.976 


.955 


.930 


.900 


.863 


.820 


.771 


.716 


.659 


.629 


.65 


.993 


.987 


.972 


.953 


.926 


.891 


.845 


.789 


.727 


.662 


.630 


.70 


.997 


.994 


.986 


.972 


.951 


.919 


.872 


.808 


.736 


.664 


.631 


.75 


.999 


.998 


.994 


.987 


.973 


.948 


.900 


.826 


.743 


.666 


.631 


.50 


1.000 


1.000 


.998 


.996 


.990 


.974 


.930 


.843 


.748 


.666 


.632 


.85 


1.000 


1.000 


1.000 


.999 


.998 


.992 


.960 


.854 


.750 


.667 


.632 


.90 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


.986 


.857 


.750 


.667 


.632 


.95 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


.999 


.857 


.750 


.667 


.632 


1.00 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


.857 


.750 


.667 


.632 



Proportion of employees considered satisfactory » .70 



.00 


.700 


.700 


.700 


.700 


.700 


.700 


.700 


.700 


.700 


.700 


1 .700 


.05 


.746 


.739 


.732 


.726 


.722 


.718 


.715 


.712 


.708 


.705 


1 .703 


.10 


.787 


.776 


.762 


.752 


.744 


.737 


.730 


.723 


.717 


.709 


1 .705 


.15 


.826 


.810 


.791 


.777 


.766 


.755 


.745 


.736 


.725 


.714 


.708 


.20 


.860 


.842 


.819 


.802 


.787 


.774 


.761 


.748 


.735 


.720 


1 .711 


25 


.890 


.871 


.846 


.826 


.809 


.793 


.777 


.761 


.744 


.725 


.714 


.30 


.916 


.897 


.871 


.849 


.830 


.812 


.793 


.774 


.754 


.731 


.718 


.35 


.938 


.920 


.894 


.872 


.852 


.831 


.810 


.788 


.764 


.737 


.721 


.40 


.956 


.941 


.916 


.894 


.873 


.851 


.828 


.803 


.775 


.743 


.724 


.45 


.970 


.958 


.936 


.915 


.894 


.871 


.846 


.818 


.786 


.749 


.727 


.50 


.981 


.972 


.953 


.935 


.914 


.891 


.865 


.834 


.798 


.755 


.730 


.55 


.989 


.982 


.968 


.952 


.934 


.912 


.885 


.851 


.811 


.761 


.732 


.60 


.994 


.990 


.980 


.968 


.952 


.932 


.905 


.870 


.823 


.766 


.734 


.65 


.997 


.995 


.989 


.981 


.969 


.952 


.926 


.889 


.836 


.771 


.735 


.70 


.999 


.998 


.995 


.990 


.982 


.970 


.948 


.909 


.849 


.774 


.736 


.75 


1.000 


1.000 


1.000 


.996 


.992 


.984 


.968 


.931 


.861 


.776 


.737 


.80 


1.000 


1.000 


1.000 


.999 


.998 


.994 


.986 


.953 


.870 


.778 


.737 


.85 


1.000 j 


1.000 


1.000 


1.000 


1.000 


.999 


.997 


.975 


.874 


.778 


.737 


.90 


1.000 ! 


1.000 , 


1.000 ! 


1.000 


1.000 


1.000 


1.000 


.992 


.875 


.778 


.737 


.95 


1.000 j 


1.000 1 


1.000 


1.000 1 


1.000 


1.000 


1.000 


.992 


.875 


.778 


.737 


i.00 


1.000 1 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


.875 


.778 


.737 



Proportion of employees considered satisfactory =* .80 



.00 


.800 


.800 


.800 


.800 


.800 


.800 


.800 


.800 


.800 


.800 


.800 


.05 


.839 


.833 


.827 


.823 


.819 


.816 


.813 


.810 


.807 


.804 


.802 


.10 


.872 


.864 


.852 


.844 


.838 


.832 


.826 


.821 


.815 


.809 


.805 


.15 


.901 


.890 


.876 


.865 


.856 


.848 


.840 


.832 


.823 


.813 


.808 


.20 


.926 


.914 


.898 


.885 


.874 


.864 


.853 


.843 


.832 


.819 


.811 


.25 


.946 


.934 


.917 


.904 


.891 


.880 


.867 


.855 


.841 


.824 


.814 


.30 


.962 


.951 


.935 


.921 


.908 


.895 


.882 


.867 


.850 


.830 


.818 


.35 


.974 


.965 


.950 


.937 


.924 


.911 


.896 


.879 


.860 


.836 


.821 


.40 


.983 


.976 


.964 


.952 


.939 


.926 


.910 


.892 


bo 

i— » 


.843 


.825 


.45 


.990 


.984 


.975 


.964 


.953 


.940 


.925 


.906 


.882 


.850 


.829 


.50 


.994 


.991 


.984 


.975 


.966 


.954 


.939 


.920 


.894 


.857 


.832 


.55 


.997 


.995 


.990 


.984 


.977 


.967 


.953 


.934 


.907 


.865 


.836 


.60 


.999 


.998 


.995 


.991 


.985 


.978 


.966 


.949 


.920 


.872 


.838 


.65 


1.000 


1.000 


1.000 


1.000 


.992 


.987 


.978 


.963 


.934 


.878 


.840 


.70 


1.000 


1.000 


1.000 


1.000 


1.000 


.994 


.988 


.977 


.948 


.884 


.841 
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N. M. Abrahams, E. F. Alf, and J. H. Wolfe 
TABLE 1 — (Continued) 






Selection ratio 



.050 


.100 


.200 


300 


.400 


.500 


.600 


.700 


.800 


.900 



Proportion of employees considered satisfactory * .80 



.75 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


.995 


.988 


.963 


.887 


1 .842 


.80 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


.999 


.996 


.977 


.888 


.842 


.85 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


.999 


.990 


.889 


.842 


.90 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


.998 


.889 


.842 


.95 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


.998 


.889 


.842 


1.00 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


.889 


.842 



Proportion of employees considered satisfactory — .90 



.00 


.900 


.900 


.900 


.900 


.900 


.900 


.900 


.900 


.900 


.900 


.900 


.05 


.928 


.924 


.920 


.917 


.914 


.912 


.910 


.908 


.906 


.903 


.902 


.10 


.949 


.944 


.937 


.932 


.928 


.924 


.920 


.916 


.912 


.907 


.904 


.15 


.966 


.960 


.952 


.946 


.940 


.935 


.930 


.924 


.918 


.911 


.907 


.20 


.978 


.972 


.965 


.958 


.952 


.946 


.940 


.933 


.925 


.916 


.910 


.25 


.986 


.982 


.975 


.969 


.963 


.957 


.950 


.942 


.933 


.921 


.913 


.30 


.992 


.989 


.983 


.978 


.972 


.966 


.959 


.951 


.941 


.927 


.917 


.35 


.996 


.993 


.989 


.985 


.980 


.975 


.968 


.960 


.949 


.933 


.921 


.40 


.998 


.996 


.994 


.990 


.987 


.982 


.976 


.968 


.958 


.940 


.926 


.45 


.999 


.998 


.996 


.994 


.992 


.988 


.983 


.976 


.966 


.947 


.930 


.50 


1.000 


1.000 


1.000 


1.000 


.995 


.993 


.989 


.984 


.974 


.955 


.935 


.55 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


.994 


.990 


.982 


.962 


.939 


.60 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


.994 


.989 


.970 


.943 


.65 


1.000 


1.000 


1.000 


1.000 


1.000 , 


1.000 


1.000 


1.000 


.994 


.978 


.946 


.70 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


.998 


.985 


I 947 


.75 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


i.ooo ; 


1.000 


.999 


.991 


' .947 


.80 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 1 


.990 


947 


.85 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 I 


.999 


| 947 


.90 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 1 


1.000 


1.000 


.947 


.95 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


.947 


1.00 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


.947 



Proportion of employees considered satisfactory * .95 



.00 


.950 


.950 


.950 


.950 


.950 


.950 


.950 


.950 


.950 


.950 


.950 


.05 


.969 


.967 


.964 


.962 


.960 


.959 


.957 


.956 


.954 


.952 


.951 


.10 


.982 


.979 


.975 


.972 


.970 


.967 


.964 


.962 


.959 


.955 


.953 


.15 


.990 


.987 


.984 


.981 


.978 


.975 


.972 


.968 


.964 


.959 


.956 


.20 


.995 


.993 


.990 


.987 


.984 


.982 


.978 


.974 


.970 


.963 


.959 


.25 


.998 


.996 


.994 


.992 


.990 


.987 


.984 


.980 


.975 


.968 


.962 


.30 


.999 


,998 


.997 


.996 


.994 


.992 


.989 


.986 


.981 


.973 


.966 


.35 


1.000 


1.000 


1.000 


.998 


.996 


.995 


.993 


.990 


.986 


.978 


.970 


.40 


1.000 


1.000 


1.000 


1.000 


1.000 


.997 


.9% 


.994 


.991 


.983 


.975 


.45 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


.997 


.994 


.988 


.979 


.50 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


.997 


.992 


.984 


.55 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


.996 


.988 


.60 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


.998 


.992 


.65 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


.999 


.995 


.70 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


.998 


.75 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


.999 


.80 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


.85 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


.90 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


.95 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.00 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 
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